Instance Embedding Transfer to Unsupervised Video Object Segmentation

We propose a method for unsupervised video object segmentation by transferring the knowledge encapsulated in image-based instance embedding networks. The instance embedding network produces an embedding vector for each pixel that enables identifying all pixels belonging to the same object. Though trained on static images, the instance embeddings are stable over consecutive video frames, which allows us to link objects together over time. Thus, we adapt the instance networks trained on static images to video object segmentation and incorporate the embeddings with objectness and optical flow features, without model retraining or online fine-tuning. The proposed method outperforms state-of-the-art unsupervised segmentation methods in the DAVIS dataset and the FBMS dataset.Comment: To appear in CVPR 201

CNN Architectures for Large-Scale Audio Classification

Convolutional Neural Networks (CNNs) have proven very effective in image classification and show promise for audio. We use various CNN architectures to classify the soundtracks of a dataset of 70M training videos (5.24 million hours) with 30,871 video-level labels. We examine fully connected Deep Neural Networks (DNNs), AlexNet [1], VGG [2], Inception [3], and ResNet [4]. We investigate varying the size of both training set and label vocabulary, finding that analogs of the CNNs used in image classification do well on our audio classification task, and larger training and label sets help up to a point. A model using embeddings from these classifiers does much better than raw features on the Audio Set [5] Acoustic Event Detection (AED) classification task.Comment: Accepted for publication at ICASSP 2017 Changes: Added definitions of mAP, AUC, and d-prime. Updated mAP/AUC/d-prime numbers for Audio Set based on changes of latest Audio Set revision. Changed wording to fit 4 page limit with new addition

Long-term modification of cortical synapses improves sensory perception

Synapses and receptive fields of the cerebral cortex are plastic. However, changes to specific inputs must be coordinated within neural networks to ensure that excitability and feature selectivity are appropriately configured for perception of the sensory environment. Long-lasting enhancements and decrements to rat primary auditory cortical excitatory synaptic strength were induced by pairing acoustic stimuli with activation of the nucleus basalis neuromodulatory system. Here we report that these synaptic modifications were approximately balanced across individual receptive fields, conserving mean excitation while reducing overall response variability. Decreased response variability should increase detection and recognition of near-threshold or previously imperceptible stimuli, as we found in behaving animals. Thus, modification of cortical inputs leads to wide-scale synaptic changes, which are related to improved sensory perception and enhanced behavioral performance

Similar Auditory Cortical Suppression by Distinct Mechanisms: Homeostasis, Inhibition, and Background Noise

The auditory cortex is critical for the understanding of speech. This task is accomplished through the nonlinear network interactions between many neurons. Cortical neurons are grouped into distinct types depending on whether they release excitatory or inhibitory neurotransmitters and molecular, biophysical, and morphological properties. Because cortical network interactions are nonlinear, perturbing these networks can produce counterintuitive results. To understand how auditory cortex accomplishes complex tasks like speech comprehension, we need to understand how nonlinearities shape network processing. This dissertation provides examples in rodent auditory cortex of manipulations that produce straightforward effects in single cells or small portions of the parameter range, but, in some cases, opposite effects in cortical networks. In chapter 1, I chronically reduced the level of inhibition in the cortex using Dlx1 knockout mice, which should expand frequency tuning in auditory cortex, but observed reduced frequency tuning. Homeostatic changes over time nonlinearly changed expansion into reduction. In chapter 2, I acutely activated two populations of interneurons that express either somatostatin or parvalbumin, which produce different forms of linear suppression in vitro, but observed the same suppression in vivo. The nonlinear elements of the recurrent cortical network obscured the type of linear suppression. In chapter 3, I added background noise, which suppress tone-evoked firing rates quasi-linearly at low intensities, but observed that noise-related suppression increased nonlinearly with noise intensity. The nonlinear mechanisms that preserve stimulus information in the presence of noise are less robust at high noise intensities. In each chapter, nonlinear effects led to unexpected results, highlighting the need to interpret results in the context of nonlinear networks to understand cortical processing